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Abstract 


While some research has suggested that video lectures are just as effective as in-person lectures to 
convey basic information to students, not everyone agrees that the flipped classroom model is an 
effective way of educating students. This research explores traditional, semi-flipped and fully-flipped 
classroom models by comparing three sections of an Introduction to Programming (Java) course that 
were taught at the same institution in Spring 2015 by three different instructors using three different 
paradigms. The data and observations collected suggests that incorporating in-class activities improves 
student satisfaction but a semi-flipped classroom, including in-class activities, some outside-class 
lecture videos, and some in-class lectures, may generally provide the best overall experience for the 
students. However, while students may be more satisfied and get more programming practice in a 
flipped paradigm, overall student performance did not appear to be greatly impacted. 

Keywords: Flipped Classroom, Inverted Classroom, Java, Introduction to Programming, Information 
Systems Education 


1. BACKGROUND 

The flipped classroom is an alternate teaching 
methodology being used in some high schools 
and college classrooms as a means of increasing 
student engagement and academic performance 
(Clark, 2015; Danker, 2015; Gunyou, 2015; 
Strayer, 2012; Vaughan, 2014). The flipped 
method operates by altering the traditional model 
of conducting lectures in the classroom, to one 
where lecture materials are distributed to 
students for their study outside of the regularly 
scheduled class meeting times, leaving in-class 
time for activities such as homework 


assignments. Lage, et al. define the 
flipped/inverted classroom as one in which the 
"...events that have traditionally taken place 
inside the classroom now take place outside the 
classroom and vice versa" (2000, p. 
32). Scheduled class times then involve students 
working individually or in groups, with computer 
and video technology on activities and 
assignments pertaining to the course subject 
matter (Bishop & Verleger, 2013; Clark, 2015; 
Gaughan, 2014; Herreid & Schiller, 2013; 
Vaughan, 2014). Findlay-Thompson & 
Mombourquette (2014, p. 64) argue that "a 
flipped classroom is most commonly described as 
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a reverse teaching model where the teacher uses 
various forms of technology such as videos to 
record the normal classroom lectures and 
students are required to view these recorded 
lectures outside regularly scheduled classroom 
time. This allows for the homework portion, or 
other interactive activities, to be completed within 
the classroom setting." Similarly Bishop & 
Verleger (2013, p. 4), suggest that the flipped 
classroom can be considered "an educational 
technique that consists of two parts: interactive 
group learning activities inside the classroom, 
and direct computer-based individual instruction 
outside the classroom." 

Prior research has suggested that video lectures 
are just as effective as in-person lectures to 
convey basic information to students (Zhang, 
Zhou, Briggs, & Nunamaker, 2006). Therefore, 
some believe that using class time to reiterate 
textbook material is not the most effective use of 
class time and may only encourage students to 
skip the reading entirely. There is an increasing 
number of faculty at various secondary schools 
and colleges that realize that the traditional 
lecture style of teaching has been ineffective in 
meeting the educational needs of students (Clark, 
2015; Gunyou, 2015; Vaughan, 2014). 

Researchers have reported a variety of positive 
aspects to the flipped model (Saulnier, 2015), 
especially in courses that can utilize lecture time 
for computer-based activities (Frydenberg, 
2013). However, some research indicates that 
there are some potential negative aspects 
(Strayer, 2012) where students reported feeling 
lost and "...were more likely to disengage with the 
material sooner than students in the traditional 
class-room" (2012, p. 189). 

2. INTRODUCTION 

This research explores the impact of three 
different teaching models (traditional, semi- 
flipped and fully-flipped classroom) in an 
Introduction to Java programming course. This 
course is required in our Computer Science major 
as well as our Computer Science and Information 
Systems minors. Each course section meets for 
3 hours per week in a lecture room (32 students 
max) and smaller groups (16 students max) meet 
for 2 hours per week in computer labs to complete 
programming activities. This research focuses on 
changing the lecture format. The labs were 
identical for all sections. 

Three sections of the Introduction to 
Programming course were taught at the same 
institution in Spring 2015 by three different 
instructors; hereafter referred to as instructor T 
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(Traditional course), instructor S (Semi-flipped 
course), and instructor F (Fully-flipped course). 
The formats were selected by the instructors 
strictly based on preference. Instructor F had 
never taught this particular course in a traditional 
format but had previously taught the course in a 
fully flipped format and had already developed 
lecture videos. Therefore, it was logical that 
instructor F teach the course in a flipped format 
again. Instructors T and S had both taught the 
course previously in a traditional format but 
instructor S was interested in trying the flipped 
format in combination with some traditional 
lecturing. This led us to a unique situation in 
which the three course sections in the same 
semester were offered in three different teaching 
styles. 

Since instructor F had taught the course most 
recently, it was agreed that all sections would use 
instructor F's course schedule so that exams, lab 
practical exams, and homework assignments 
could be similar across the sections. Unlike 
previous offerings of the course, students were 
allowed to select any lab/lecture combination. 
This meant they did not necessarily have the 
same lecture instructor as they did for lab. Tests 
were given during lab time to give students 
enough (and equal) time to complete the test (2 
hours), since two sections had lectures twice a 
week (85 minutes each class session) and one 
(instructor T) met three times each week (60 
minutes each class session). This made it 
critically important that all sections of the course 
stay on the same schedule, covering the same 
material at the same time. 

The exams and lab practical exams were 
developed collectively among the instructors and 
were all very similar but not identical since the 
students were not all taking them at the same 
time. The five homework assignments and the 
final exam were also developed collectively and 
were identical for all sections of the course. The 
first homework assignment was similar to the in- 
class activities, including several small exercises 
to practice basic skills. The last four homework 
assignments were larger projects that allowed 
students to use the knowledge they gained to 
develop working games. 

Students were unaware of the format differences 
prior to the start of the class so there was no self¬ 
selection for a particular teaching style during 
registration. Additionally, no students changed 
sections after the classes began. 
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3. METHODOLOGY & FINDINGS 

Traditional Classroom (T) 

Instructor T planned on delivering traditional 
PowerPoint and whiteboard-based lectures, with 
some class discussion and quizzes each week 
based on the text readings. Since no in-class 
activities were planned for this section of the 
course, Instructor T was not assigned any 
teaching assistants (TA's). Instructor T's section 
included 13 students with 8 CS majors, 1 CS 
minor and 1 Physics/Computational Science 
major. 

About three weeks into the course, a major 
problem emerged: since most of the class time 
was spent on lecture, material was covered much 
faster than the fully- and semi-flipped lecture 
sections. Instructor T was already a chapter and 
a half ahead of the other sections and it was 
impossible to proceed at this pace since all 
instructors agreed ahead of time to give our 
students the same exams and the same labs. 

Initially, the students seemed to be comfortable 
with the pace of the traditional course - two of 
the 13 students had a Java class in high school. 
However, the quiz results were disappointing - 
probably because students were not doing the 
reading assignments. 

Before this semester, instructor T felt that 
students were getting enough practice 
programming through the labs and the five 
programming assignments given throughout the 
semester. Instructor T was compelled to shorten 
the lecture component and add a programming 
activity to each class session - this is what 
enabled the course to synchronize with the other 
course sections. While every student owned a 
laptop, sometimes they forgot to bring them or 
their laptop was being repaired. Fortunately, the 
department has five "loaner" laptops readily 
available. 

The first in-class programming activity was end- 
of-chapter exercises as sometimes employed by 
the other instructors, but instructor T designed 
the rest to match the important points presented 
during the shortened lecture. Some activities built 
upon previous activities. All of the students 
immediately liked the programming activity, 
(student survey indicated 9.75 out of 10 - 
strongly agree that "in-class programming 
exercises improved my comprehension of the 
course material") even staying after class to 
complete them, and instructor T could give 
immediate help to struggling students. Students 
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indicated that these should have counted for 
more of their grade, and instructor T agrees. 

Another successful class activity was to play "The 
Good, the Bad, and the Ugly." The instructor 
selected excellent programs from labs or 
programming projects (the good), programs that 
did not work or had major problems (the bad), or 
mostly worked but were inefficient or hard to 
follow (the ugly). Student's names were removed 
from the code. The programs were projected on 
the screen, and students were invited to choose 
what category each example was from. The class 
would discuss why it was in that category, and 
explore ways to improve the code. Students 
actually requested this activity several times. 

In previous semesters, there were a few students 
that failed to hand in programming assignments, 
usually resulting in lack of engagement and poor 
grades. To remedy this problem, instructor T 
would allow students to start their projects in 
class in place of the regular programming activity. 
This enabled instructor T to answer questions 
about the project that students encountered as 
they began coding. This resulted in less panicked 
students coming to office hours the day before 
the project was due, resulting in less stress for all 
involved. 

A downside to having to readjust lecture time for 
programming activity was that the lectures were 
not as organized as they could have been; as 
recorded lectures for flipped classes tend to be 
carefully structured. However, it was liberating to 
be able to (mostly) abandon the slides and 
concentrate on key concepts. 

In the future, instructor T plans to add in-class 
programming activities to other programming 
courses, along with fill-in worksheets to guide 
note taking during lectures. 

While the traditional section did include the 
activity-based learning associated with a flipped 
lecture, this section did not include the video 
lectures of the fully- and semi-flipped sections, 
which enabled instructors S and F to devote even 
more lecture time to activities. Table 1 compares 
student evaluation scores for the last time 
instructor T taught the course (i.e. Fall 2005) 
versus the recent (i.e. Spring 2015) course. 
Instructor T did not teach this course between 
those two semesters (a 10 year gap) and thus, 
did not have the opportunity to refine or improve 
the course. 
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Measure 

S 2015 

S 2005 

Classes helpful 

7.8 

6.8 

Classes interesting 

7.5 

6.7 

Course rating 

8.1 

6.8 


Table 1. Instructor T Student Evaluation Scores 


Semi-Flipped Classroom (S) 

In 2010, instructor S was not satisfied with his 
student evaluations from this particular course 
and wished to improve student engagement. 
However, instructor S could not devote the time 
necessary to entirely flip the course and sought 
to create one video and in-class activity each 
week. 

Instructor S's section included 33 students with 
24 CS majors, 3 CS minors and 6 IS minors. This 
section's lecture was scheduled on Tuesdays and 
Thursdays for 85-minute sessions. 

Instructor S narrated and produced videos using 
Camtasia that included the presentation of lecture 
slides and code demonstrations related to the 
assigned textbook reading for the week. While the 
videos were designed to represent what might 
happen in a single 85-minute lecture session, 
editing out pauses and the lack of student 
interaction allowed the information to be 
presented more efficiently in videos that ranged 
from 24 minutes to 50 minutes (45 minutes on 
average). Typically, the videos would be posted 
to the course website the Friday before a Tuesday 
activity session. 

The instructor was able to create 9 videos and 10 
in-class activities. During 10 of the 13 Tuesday 
lecture sessions, students were required to bring 
a laptop computer to take an online quiz using the 
Blackboard LMS and then complete a 
programming activity based on the reading 
assignment and video posted on Friday (see 
Appendix A for sample quiz questions and activity 
instructions). Students were asked prior to the 
first class if they had a laptop available to bring 
to class. All students reported that they did. 

Each quiz consisted of 10 questions (mostly 
multiple choice). Students were given 10 minutes 
to complete the quiz then immediately begin a 
programming activity designed to fill the 
remaining 75 minutes of class time. The activity 
included a deliverable (typically a small working 
program) submitted to the LMS. The instructor 
was assigned two undergraduate teaching 
assistants with Java programming experience to 
help answer students' questions. During the 
semester, instructor S deployed 9 videos and 
administered 10 quizzes and 10 activities. During 
one week, the quiz and activity was based on the 


reading, as the instructor could not deploy a video 
in time. Three Tuesday lectures were used to 
review for exams or to give students special help 
with homework. 

On Tuesday activity sessions, the instructor would 
use the podium computer and the LMS to monitor 
students' quiz grades as they were completed. 
Two to three quiz questions were designed to be 
trivially easy for those who watched the video. If 
a student got all the trivial questions wrong, the 
instructor would dismiss the student to a nearby 
lounge to review the reading and re-watch the 
video. Three students were dismissed twice in the 
first three weeks of the course. By the fourth 
week, these three students were getting the 
trivial questions correct. Two additional students 
were dismissed just once early in the semester. 
In general, the quiz setup proved valuable in 
immediately identifying students who were not 
watching the videos and helped correct this 
deficiency in a positive way. Students who did 
poorly on the quizzes were dismissed and lost 
valuable lecture activity time to get help from the 
instructor and student assistants. However, as 
long as students returned in a timely fashion, 
worked until the end of the period and submitted 
the activity by the next lecture period (Thursday), 
no penalties were given. 

Thursday sessions were used for traditional 
lecturing, which instructor S used exclusively 
when teaching the course in Fall 2010. Traditional 
lecturing includes the presentation of lecture 
slides and programming demonstrations. By 
replacing 10 out of 25 (40%) lecture sessions 
with activities, instructor S yielded improvements 
in student evaluations. Table 2 compares student 
evaluation scores for the last time instructor S 
taught the course (i.e. Fall 2010), which was in a 
traditional format, versus the recent (i.e. Spring 
2015) semi-flipped course. Instructor S did not 
teach this course between those two semesters. 
This change in student evaluation scores is the 
most significant improvement instructor S has 
achieved in 12 years of teaching. It is unlikely 
that such improvement is the result of simple 
instructor maturation. 


Measure 

S 2015 

F 2010 

Classes helpful 

9.1 

8.3 

Classes interesting 

8.9 

7.5 

Course rating 

9.4 

7.4 


Table 2. Instructor S Student Evaluation Scores 


In addition to these improvements, a majority of 
the semi-flipped classroom students agreed 
(9.2/10 on Strongly Disagree to Strongly Agree 
Likert-scale) with the statement, I like the 
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"flipped classroom" teaching model, i.e. Watching 
the lecture videos outside of class and spending 
more class time doing team exercises. One 
student commented "I really found the 
video/activities helpful" (see Appendix A, Table I 
for complete comment summaries). But, not all 
students prefer the format (see Appendix A, Table 
II). One student stated "I really did not care for 
the videos at all. They really were just difficult to 
focus on and weren't very helpful." A vast 
majority of the 33 students strongly agreed 
(9.5/10 on Strongly Disagree to Strongly Agree 
Likert-scale) with the statement: The in-class 
programming exercises improved my 
comprehension of the course material. Not 
surprisingly, students tended to disagree (5.3/10 
on Strongly Disagree to Strongly Agree Likert- 
scale) with the statement, Prefer normal course 
over flipped. 

Fully-Flipped Classroom (F) 

The fully-flipped section had 16 students, only 4 
of which were CS majors. The instructor had 
created 54 lecture videos, each of which is 
approximately 10 minutes in length. The lecture 
videos were created using Camtasia. The videos 
included lecture narration of PowerPoint slides, 
coding demonstrations in BlueJ, and interactive 
quizzes. The quizzes did not count towards the 
students' grades; their purpose was to help the 
students assess their understanding of the 
concepts as they progressed through the videos. 
The flipped section's lecture was scheduled on 
Mondays and Wednesdays for 85-minutes each 
session. Before each class, students were 
required to watch ~2 lecture videos and read the 
related textbook sections. Finally, in addition to 
their textbook, students were required to bring a 
laptop to class. Similar to the semi-flipped 
section, students were asked prior to the first 
class if they had a laptop available to bring to 
class. All students reported that they did but 
again the department had "loaner" laptops 
available. 

Instructor F used two techniques to strongly 
encourage students to watch lecture videos. 
First, students were required to enter their name 
and email address in order to watch the lecture 
videos. The instructor was then emailed a 
detailed report as to how much of the content 
each student watched. Students can play the 
video in the background and turn the sound off so 
certainly this is imperfect. Secondly, the videos 
contained secret phrases, which had to be 
submitted via a Blackboard quiz. The phrase 
could be given verbally or written on the slides 
and could change several times throughout the 
video; the intent being to check if students were 


14(5) 
September 2016 

actually paying attention to the video. Of course, 
students could share the phrase with each other 
so this is not foolproof either. 

Each class session would begin by instructor F 
soliciting the students for questions regarding the 
lecture video and readings, which typically took 
only a few minutes. Once all questions were 
answered, the students took a short online quiz 
(via Blackboard LMS) to assess their knowledge 
based on the lecture videos and readings. 
Quizzes were automatically graded and solutions 
were given by the LMS. Students were 
encouraged to ask questions regarding the quiz. 
Quizzes were fairly low-stakes, similar to 
Bormann (2014). The quizzes only counted for 
10% of their final grade and the 3 lowest quizzes 
were dropped. However, students were unaware 
that quizzes would be dropped until the last 2 
weeks of the semester. The intention was to 
motivate them to watch the lecture videos and 
learn from the mistakes they made in the quizzes. 

After completing the quiz and getting answers to 
their questions, the students would begin the in- 
class activity. The class activities were directly 
related to the concepts covered on that day's 
lecture videos and readings. Students were 
encouraged to work with their fellow classmates 
as they worked through the in-class activities. In 
addition to the instructor, a sophomore TA, that 
had taken the course last year, was available to 
help answer questions during class. The TA 
would complete the in-class activities a day or two 
before each class to prepare. Both the instructor 
and the TA would move about the room 
answering questions and checking on 
shyer/quieter students. 

The class activities were developed to take the 
average student the entire class time to 
complete. Some students did require more time 
so all students were given until the beginning of 
the next class to submit the solution. All 
assignments were submitted via the LMS and 
graded within a few days so the students had 
quick and frequent feedback. Stronger students 
that could complete the activities quickly were 
allowed to leave if they completed all their work 
and submitted it for grading. In some cases, more 
challenging extra credit activities were offered as 
an option if they wanted to stay or they were free 
to use the time to work on homework. Some 
would even stay to help their fellow classmates 
work through the activity. This was encouraged 
but closely monitored to make sure the help was 
in the form of guidance not simply providing the 
solution. The only restriction was that students 
were not allowed to help each other with 
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homework; all homework questions needed to be 
asked of the instructor or an official departmental 
tutor. 

Instructor F had taught this course the previous 
year in a fully-flipped format with zero in-class 
lecturing, other than answering questions. 
Instructor F was concerned that students may not 
be as connected with the instructor as they would 
be in a more traditional setting. Therefore, the 
first three weeks of the Spring 2015 course, 
Instructor F began each class with ~15 minutes 
of lecturing to supplement the lecture videos. 
During the fourth week, students were anxious to 
jump right into the class activities so the 
instructor decided to go back to the fully-flipped 
format, only answering questions at the 
beginning of class and discontinuing traditional 
in-class lecturing. The only exceptions were two 
exam reviews. Students were given a practice 
exam to complete several questions and then the 
instructor led a short review/discussion of the 
solutions. 

A majority of the fully flipped classroom students 
agreed (8.93/10 on Strongly Disagree to Strongly 
Agree Likert-scale) with the statement, I like the 
"flipped classroom" teaching model, i.e. Watching 
the lecture videos outside of class and spending 
more class time doing team exercises. One 
student reported the following: "I absolutely 
LOVE the flipped classroom. The lectures outside 
of class were short, sweet, and to the point and I 
actually enjoyed going to class. The class 
activities were extremely helpful in understanding 
the material" (also see Appendix A, Table I). 
Nonetheless, not all students prefer this method 
of learning (see Appendix A, Table II). As one 
student stated on the evaluation, "I didn't like the 
reversed classroom experience. I much rather 
prefer to listen to a lecture in class and then do 
the homework on my own time." A majority of 
students strongly agreed (9.71/10 on Strongly 
Disagree to Strongly Agree Likert-scale) with the 
statement: The in-class programming exercises 
improved my comprehension of the course 
material. 

The frequent quizzes were helpful in assessing if 
students are watching the lecture videos and 
completing the readings. This was also a good 
way to give immediate feedback to students 
regarding their progress. Flowever, the volume 
of quizzes can be overwhelming to the students. 
As one student commented, "I did not like that 
we had so many quizzes. We had them in the 
required lecture videos, after the lecture videos, 
and in class almost every day. At times it seemed 
like too much." Nonetheless the same student 


14(5) 
September 2016 

acknowledged the benefit by adding, "However, I 
do think they were helpful (just tedious)". Table 
3 shows student evaluation data for the two 
semesters where instructor F taught the course in 
a flipped format. 


Measure 

S 2015 

S 2014 

Classes helpful 

9.4 

8.86 

Classes interesting 

9.27 

8.95 

Course rating 

9.47 

9.29 


Table 3. Instructor F Student Evaluation Scores 


The Spring, 2014 section had zero in-class 
lecturing, while the Spring 2015 had 8% of its in- 
class time devoted to more traditional lecturing. 
This included the 15 minute sessions at the 
beginning of class during the first 3 weeks of the 
course and the exam review sessions. 

Another item to note is that the Spring 2015 
section had a high number of IS students versus 
CS students. Many of these students in particular 
expressed concern about the difficulty of the 
course material during the first week, but after a 
few weeks of in-class coding practice they 
seemed to gain confidence and ultimately did 
quite well. Four of the five A's on the final exam 
were achieved by non-CS majors. 

There may be some value in doing some 
traditional lecturing during the first few weeks of 
a flipped classroom environment to connect with 
the students. While more empirical data is 
needed to confirm this, instructor F did feel more 
connected with the students in the Spring 2015 
course. Having a short lecture at the beginning 
of each class during the first 3 weeks allowed the 
instructor to learn all the students' names and get 
a better sense of their abilities. This also likely 
allowed the students to get to know their 
instructor better and perhaps made them more 
comfortable asking questions. 

General Observations & Challenges 

The three instructors agree that the flipped model 
slows down the pace of the course. In practice, 
the flipped classroom students spend more time 
on in-class activities than listening to lecture 
making it difficult to cover as much content as a 
traditional course. While in principle one can add 
more video lectures to cover more content, this is 
difficult to achieve for many reasons outlined 
below. In practice, our observation was that 
students in the traditional lecture get exposed to 
more content and the flipped students get less 
exposure but get more practice programming. 

Videos are extremely time-consuming to make. 
It was not uncommon for it to take 5-6 hours to 
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create a 10-minute video. Videos have no value 
unless students watch them. While students can 
also miss lecture, absences can be precisely 
measured whereas measuring whether or not 
students watch the videos is challenging. 
Instructors F and S used various techniques 
described in the previous section to mitigate this 
issue but no technique was perfect. 

Videos can be more engaging than reading the 
textbook and embedded quizzes can help verify 
that a video is being watched, but ultimately 
there is no way to verify that students are 
thoroughly watching the videos. Similar to text 
book reading, the benefits depend on the 
individual learner. 

If the fully- and semi-flipped students watch all 
the videos and typically use the entire class 
period or more to complete the in-class activities, 
they could have more "seat time" than students 
in a traditional setting. In other words, the explicit 
time spent on lecture-related activities could be 
higher. These students definitely get more 
programming practice, in the form of in-class 
activities, than traditional students typically get 
but it is not necessarily the case that they are 
spending more time on the course. For instance, 
some students were able to complete in-class 
activities quickly and either leave class early or 
move on to other tasks (e.g. homework or 
watching lecture videos). It is also certainly the 
case that some students simply did not watch the 
lecture videos, reducing their "seat time". Finally, 
unlike traditional classrooms, students were 
given time in-class to work on homework 
assignments. Since they are able to get help from 
their instructor during this time, this could have 
significantly reduced the amount of out of class 
time students spent on these assignments versus 
traditional classroom students. 

Whether or not fully- and semi-flipped students 
are actually getting more seat time than 
traditional students depends on a variety of 
factors such as lecture video length, student 
willingness to watch the videos, and student 
ability. Nonetheless, fully- and semi-flipped 
students may perceive that they have more seat 
time than a traditional class and may not like it. 
But, the student evaluation data and comments 
did not reveal frustration with increased time 
commitment. However, some students did report 
not liking the lecture videos (see Appendix A, 
Table II). Some of this could be related to the 
length of the videos. Instructor F's videos were 
typically ~10 minutes each, while instructor S's 
videos were 40 minutes on average. 
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Perhaps most importantly, the lecture format did 
not have an impact on students' performance on 
the final exam, which was identical for all 
sections. Table 4 shows how close the mean final 
exam scores were for students in the three 
different lecture sections. 


Instructor F 

Instructor S 

Instructor T 

84.3 (n = 16) 

84.5 (n = 33) 

83.4 (n = 13) 


Table 4. Mean Final Exam Scores 


4. CONCLUSIONS & FUTURE WORK 

There can be an incredible start-up cost to 
creating flipped classroom materials. Creating the 
lecture videos is very time consuming but they 
can certainly be reused. Bergmann (2012) 
recommends short interactive videos that are IV2 
minutes in length per grade level -- under 20 
minutes for an introductory college level course. 
While some instructors may use generic lecture 
videos to flip their classroom, Sams and 
Bergmann (2012) recommend that instructors 
create their own. One possible danger is tying 
lecture videos too closely to a specific textbook 
(e.g. using textbook slides). This could make it 
difficult to switch textbooks or even go to a newer 
edition. 

Our comparative evaluation of flipped versus 
traditional was consistent with previous work. We 
observed that flipped learning provides students 
opportunities to learn in a more differentiated 
manner than traditional linear and passive forms, 
which is consistent with Willey and Gardner 
(2013). We observed and informally measured 
that the vast majority of students completed the 
required prerequisite tasks on a fairly regular 
basis while only a very small portion did not, 
which is consistent with Davies et al. (2013), 
Gaughan (2014), and Murphree (2014). 

The semi-flipped model is an effective 
compromise for an instructor who cannot devote 
the time needed to replace all lectures with 
videos. Based on our observations, the semi- 
flipped model produces similar positive 
impressions from students without the need to 
replace all traditional lecturing. However, 
instructor S notes that early in the semester 
students were confused about the format and it 
took nearly 3 weeks before the vast majority of 
students were coming to the activity session 
prepared, whereas, instructor F reports a quicker 
turnaround time with respect to student comfort 
and preparation. 

We observed that flipped learning empowered 
students through more active learning, which is 
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consistent with (Lage et al., 2000). Specifically, 
students were held more accountable for studying 
material prior to coming to regular lectures. 
However, while we can conjecture that students 
in the semi- and fully-flipped sections may have 
more programming practice, and should therefore 
perform better when assessed, the average final 
exam scores shown in Table 4 indicate no 
significant difference in student performance 
based on the lecture format. More generally, 
whether or not semi-flipped yields a better 
understanding of the course material is yet to be 
determined. 

It is also unknown what balance of flipped versus 
traditional works best and how to structure that 
balance. Instructor S had one day a week 
dedicated to lecture and one for in-class activities 
whereas Instructor F only did some short 
lecturing at the beginning of the semester and for 
exam review. While instructor T initially planned 
to stick with traditional lecturing in the classroom, 
the schedule forced some adjustment including 
incorporating programming activities during 
lecture time. Ultimately, this addition was well 
received by both the students and instructor. 

The flipped model, whether full or semi, creates 
certain advantages that are difficult to achieve in 
a traditional model. For example, in a lecture only 
model, it is often not revealed that a student is 
lost until the first homework assignment or even 
the first exam. Doing poorly on these higher 
stakes items can have devastating effects on 
some students, crushing their confidence and 
willingness to keep trying. In a flipped or semi- 
flipped classroom, students can experience 
weekly or even daily feedback via low-stake 
activities and quizzes. This enables both students 
and instructors to get early and frequent 
feedback. Students who stumble along the way 
on low-stakes deliverables still have time to 
recover before higher stake exams. And, 
instructors can detect problems early and 
intervene quickly to get students back on track. 
In addition, as students work through the in-class 
exercises, the instructor can also detect students 
that need extra attention and identify general 
pain points for all students, which can then be 
addressed quickly via a short ad-hoc lecture or a 
supplemental lecture video. 

While the quizzes and questions that encourage 
video watching are low-stakes with respect to 
grading, there are still consequences that 
encourage students to take them seriously. Many 
students naturally want to make a good 
impression with the instructor and are 
embarrassed if it is revealed that they are not 
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prepared. When a student does poorly on a major 
test, it can be unclear to both the instructor and 
student if the poor performance is the result of a 
lack of understanding or a lack of preparation. 
Whereas, when a student fails to answer a trivial 
question that was directly answered in a video, 
the source of the student's problem is clearer to 
both student and instructor. 

Another major advantage of the flipped classroom 
is that strong students are not held back during 
class activities. They are free to work as quickly 
as possible and, at least in the case of these 
course offerings, were allowed to leave if they 
completed all their work. 

Based on our observations and survey results, the 
in-class activities associated with the flipped 
model had the most profound impact in improving 
students' impressions of the course, whereas the 
videos themselves were not viewed as 
positively. Thus, improvements can be achieved 
without flipping an entire course as instructor T 
and S learned by adding in-class activities without 
completely replacing traditional lectures with 
video. But, using lecture time for activities 
certainly decreased the amount of material that 
was covered in the course. This was true even 
when videos were used extensively. While video 
lectures can substitute for traditional lectures, in 
practice, this is challenging to achieve because 
some students may not learn the material as 
effectively from video lectures. 

Since some students will prefer traditional 
lecturing, it might make sense to not only offer 
the different sections but also advertise them as 
such. This would allow students to self-select the 
teaching style that best fits their needs. Future 
research might include assessing student 
satisfaction after self-selection. 
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Appendix A - Sample Quizzes and Activities 


Sample Quiz Questions from Semi-Flipped Class (week 2) 

Question Consider these four lines of code: 

a. JPanel panel = new JPanelQ; 

b. Imagelcon me = new Imagelcon("eric.jpg"); 

C. JLabel image = new JLabel(me); 

d. panel.add(image) ; 

Which line of code (a. b. c or d) uses a default constructor to create a new object with default 
values? 

Question Consider these four lines of code: 

a. JPanel panel = new JPanelQ; 

b. Imagelcon me = new ImagelconC'eric. jpg") ; 

C. JLabel image = new JLabel(me); 

d. panel.add(image); 

Which line of code (a. b, c or d) uses a default constructor to create a new object with default 
values? 


Sample Activity Segment from Semi-Flipped Class (week 2) 


1 . 

2 . 

3. 

4. 

5. 

6 . 

7 . 


Download BasicProqrams.zip and save to an appropriate folder on your computer. 
Unzip/extract the BasicPrograms folder. On a PC, right-click the zip file and select 
a MAC, double-click the zip file to extract the folder. 


Extract All 


On 


Project --> Open Project 


and then find 


In BlueJ, open the BasicPrograms project. Select 
the BasicPrograms project folder. 

Modify the AdLib program so that it creates a humorous "ad lib" story. AdLib example 

You should prompt the user for at least 7 words (nouns, verbs, adjectives, etc.) and output at least 

two sentences. Be creative and have fun, but do not spend more than 25 minutes on this part. 

You should prompt the user for a particular word using JOptionPane. showlnputDialog and 
store the word using a String variable. 

You should output each sentence with a separate System.out.println statement. 









Information Systems Education Journal (ISEDJ) 
ISSN: 1545-679X 


14(5) 

September 2016 


Sample Quiz Questions from Semi-Flipped Class (week 11) 

If you did not know the exact size of an array called nums, how could you access the last value? 

nums[nums.length] 

nums[nums.size] 

nums[nums.length-1) 
nums[nums.size-1) 

Which code shows how to declare an array called nums for storing 10 integer values? 
int nums = new array[10); 

O intQ nums = new int[10); 

int[10] num§ = new intQ; 

int array[10] nums; 

Sample Activity Segment from Semi-Flipped Class (week 11) 

1. In the array project, create a new class called if statement 

2. Replace all the code inside the class with one main method that does the following: 

o Prompt the user to type the temperature and their mood 
o Store the temperature as a double and the mood as a String 

o Print "play" if the user's mood equals "happy" and the temperatures is between 50 and 90 
(inclusive) 

o Print "play" if the user's mood is not equal to "happy" and the temperatures is between 75 
and 85 (not-inclusive) 

o Print "stay inside" if the user's mood equal "sad" or "average" and the temperatures is 
below 75. 

o Print "unsure" if all of the above conditions are false. 


Question 

Answer 


Question 

Answer 
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Appendix B - Student Evaluation Open-Ended Questions 


Table I: Liked about course/instructor 


Traditional 

• Fie was enthusiastic about the class and took the time to explain how to 
make our code better. Fie would help us with any problems on our activities 
as well and explain the solutions thoroughly. 

• The in class activities were key. That's where I learned most of the material. 

• I enjoyed the activities that the professor gave at the end of class. I found 
that they really helped me learn the material. 

• I think this way of running the class was effective. 

Semi-Flipped 

• Interacting and working with classmates at least once a week was helpful 
and productive. 

• I really found the video/activities helpful and his attitude toward wanting 
each individual student to succeed. 

• I liked the activity sessions of the course, and I thought that they were very 
helpful. 

• I think that the course was fine. I liked the lecture style of the instructor 
and it helped me a lot. 

• I learned a lot despite the incredibly fast pace. If I didn't understand 
something, the assignments were designed to let me figure it out on my 
own. 

• the homework. I like to do the homework 

Fully-Flipped 

• exercises in class helped alot 

• I really enjoyed the flipped classroom setting. The videos gave me the right 
amount of information and then being able to go right into class and do the 
exercises helped to make the concepts clearer. Also hiving the time do 
homework in class was very helpful, the homework was challenging, and 
having the ability to ask questions made it less frustrating 

• I liked that the class is completely hands on, with this type of course it is 
extremely helpful to be constantly practicing the material because being 
lectured on it won't make sense unless you are actually being challenged to 
figure out what the material means. 

• I liked that it was a lot of hands-on activities rather than lectures, I think it 
suits well with the course material. 

• I enjoyed the ... and the flipped classroom 

• The homework assignments were the best part of the class because it really 
tested you on the material. 

• I really liked the format. Watching the videos before class and then getting 
straight to work. In practice it was kind of like having two lab classes. 

• I absolutely LOVE the flipped classroom. The lectures outside of class were 
short, sweet, and to the point and I actually enjoyed going to class. The 
class activities were extremely helpful in understanding the material. 
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Table II: Disliked about course/instructor 


Traditional 

• I did not like the quizzes we had in the beginning, those were tough. Also, I 
wish the activities counted for more than just participation points. 

• Progressed through concepts slowly. 

• Sometimes the lectures seemed unorganized but I think that is because we 
didn't go over every slide and instead skipped around. All the important 
material was still tauqht so it was not much of a problem. 

Semi-Flipped 

• Online videos were an interesting idea but it was difficult to understand the 
concepts at times and the quizzes were sometimes difficult. 

• I really did not care for the videos at all. They really were just difficult to 
focus on and weren't very helpful. 

• I also feel like there was too much material in the traditional lectures. I 
think that watching someone code for an hour and a half doesn't really help 
me at all. 

• I think that it would have been more helpful if the material had been 
presented and then we worked with partners or in small groups to practice 
the code. 

• There definitely needs to be traditional lectures to introduce things like 
arrays, Java in general, etc, but I really think that watching someone else 
doing a proqram really isn't practicing anythinq. 

Fully-Flipped 

• videos outside of class 

• I did not like that we had so many quizzes. We had them in the required 
lecture videos, after the lecture videos, and in class almost every day. At 
times it seemed like too much. However, I do think they were helpful (just 
tedious). 

• I didn't like the reversed classroom experience. I much rather prefer to 
listen to a lecture in class and then do the homework on my own time. 
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